An Improved Algorithm for the Solution of the Regularization Path of SVM

نویسندگان

  • Chong Jin Ong
  • Shi Yun Shao
  • Jian Bo Yang
چکیده

This paper describes an improved algorithm for the numerical solution to the Support Vector Machine (SVM) classification problem for all values of the regularization parameter, C. The algorithm is motivated by the work of Hastie et. al. and follows the main idea of tracking the optimality conditions of the SVM solution for descending value of C. It differs from Hastie’s approach in that the tracked path is not assumed to be one-dimensional. Instead, a multi-dimensional feasible space for the optimality condition is used to solve the tracking problem. Such a treatment allows the algorithm to handle data set having duplicate points, nearly duplicate points and linearly dependent points, a common feature among many real-world data sets. Other contributions of this paper include a unifying formulation of the tracking process in the form of a linear program, an initialization routine that deals with duplicate data points, a routine that guards against accumulation of errors resulting from the use of incremental updates, a heuristics-based routine and update formula to speed up the algorithm. The algorithm is implemented under the Matlab environment and tested with several data sets including data set having up to several thousand data points.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

PSO-Optimized Blind Image Deconvolution for Improved Detectability in Poor Visual Conditions

Abstract: Image restoration is a critical step in many vision applications. Due to the poor quality of Passive Millimeter Wave (PMMW) images, especially in marine and underwater environment, developing strong algorithms for the restoration of these images is of primary importance. In addition, little information about image degradation process, which is referred to as Point Spread Function (PSF...

متن کامل

A numerical approach for solving a nonlinear inverse diusion problem by Tikhonov regularization

In this paper, we propose an algorithm for numerical solving an inverse non-linear diusion problem. In additional, the least-squares method is adopted tond the solution. To regularize the resultant ill-conditioned linear system ofequations, we apply the Tikhonov regularization method to obtain the stablenumerical approximation to the solution. Some numerical experiments con-rm the utility of th...

متن کامل

Characterizing the Solution Path of Multicategory Support Vector Machines

An algorithm for fitting the entire regularization path of the support vector machine (SVM) was recently proposed by Hastie et al. (2004). It allows effective computation of solutions and greatly facilitates the choice of the regularization parameter that balances a trade-off between complexity of a solution and its fit to data. Extending the idea to more general setting of the multiclass case,...

متن کامل

Near-Minimum-Time Motion Planning of Manipulators along Specified Path

The large amount of computation necessary for obtaining time optimal solution for moving a manipulator on specified path has made it impossible to introduce an on line time optimal control algorithm. Most of this computational burden is due to calculation of switching points. In this paper a learning algorithm is proposed for finding the switching points. The method, which can be used for both ...

متن کامل

Automatic estimation of regularization parameter by active constraint balancing method for 3D inversion of gravity data

Gravity data inversion is one of the important steps in the interpretation of practical gravity data. The inversion result can be obtained by minimization of the Tikhonov objective function. The determination of an optimal regularization parameter is highly important in the gravity data inversion. In this work, an attempt was made to use the active constrain balancing (ACB) method to select the...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2009